Imaging systems exist that use two or more separate imagers to capture two or more separate images or video streams of a target object or scene. For example, a multimodal imaging system (also referred to as a multispectral imaging system) that comprises at least two imaging modules configured to capture images in different spectra (e.g., different wavebands) is useful for analysis, inspection, or monitoring purposes, since a same object or scene can be captured in images of different spectra that can compared, combined, or otherwise processed for a better understanding of the target object or scene.
However, due to different field-of-views (FOVs), different orientations, and/or the relative physical displacements between the two or more separate imagers in such systems, images captured by one imaging module may show different portions of a target object or scene and/or show the target object or scene in different positions and sizes than images captured by other imaging modules of the imaging system. In other words, the two or more separate image or video streams captured by such systems are misaligned, which makes it difficult to combine, compare, or otherwise process or analyze the separately captured image or video streams.
Conventionally, registration (e.g., alignment) of such separately captured image or video streams is achieved by mechanically aligning (e.g., fixing the physical displacement and positions of) the two or more imagers under a very tight tolerance, manufacturing optical elements and imaging sensors under a very tight tolerance, and by calibrating the imaging modules/devices that are manufactured and aligned under the tight tolerance such that one or more of the separately captured image or video streams can be image-processed according to the calibration to obtain registered images.
However, such tightly controlled mechanical alignment, manufacturing, and calibration are costly, not always available, not stable, or otherwise impractical. Obtaining or maintaining registration calibration is problematic for multimodal imaging systems because the separately captured images are of different modality, and especially problematic for imaging systems in which a user is given the freedom and flexibility to add or replace imagers, such as for example by attaching a device attachment having a specialty imager to a mobile device equipped a digital camera, because the mechanical alignment of the different imagers is not known, not stable, and/or not controlled at the time of the production.
Accordingly, there is a need for an imaging system that can dynamically co-register two or more image/video streams, captured by separate imagers, including imagers of different modality, even when the mechanical alignment of the different imagers is not known, not stable, and/or not controlled at the time of the production.